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DETAILED ACTION 

Response to Remarks 

The Office Action has been issued in response to amendment filed January 19, 
2007. Claims 1-19 are pending. Applicant's arguments have been carefully and 
respectfully considered in light of the instant amendment, and are not persuasive. 
Accordingly, this action has been made FINAL. 

Specification Objection 

The specification has been has been amended to remove a hyperlink and 
corrected the misspelled word "spot". Therefore the objection has been removed. 



Response to Rejection Arguments 
In response to applicant arguments on page 8-9, [p][2-4] that the references 
failed to teach or show comparing the identified movement in the selected critical area 
with a database associates movements in selected areas with specific physical 
condition, the Examiner disagrees with Applicant. Pantic clearly teaches comparing the 
identified movement in the selected critical area with a database associates movements 
in selected areas (specific facial actions; page 881 , [p][3], line 10) with specific physical 
condition (physiological reaction, page 881, [p][3], line 13). The physiological reaction 
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reactions give insights as to a person physical condition, for example, if someone is 
sweating, this could be an indication that the person has a fever or very hot. 

Applicant has also argued that the Vining reference lacks motivation (see page 9- 
10, [p][5-9]) and the Examiner applied improper hindsight to combine the teachings of 
Vining, Black and Pantic to meet the limitation generating a report of the emotional and 
physical condition of the subject, the Examiner disagrees. Note that both Pantic and 
Black produces an output as to the condition of an individual (see Fig 9, of Black and 
see Fig 8 of Pantic), however, neither implicitly outputs produces a "report". Vining's 
reference is directed a method of reporting findings (column 2, line 11-12) of an expert 
analysis of image data that includes the step of. generating a report (column 3, line 37- 
38). Therefore, Vining's reference would cure the deficiencies of Black and Pantic by 
producing a formal report as to the emotional and physical conditions of an individual. In 
response to applicant's argument that the examiner's conclusion of obviousness is 
based upon improper hindsight reasoning, it must be recognized that any judgment on 
obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. 
But so long as it takes into account only knowledge which was within the level of 
ordinary skill at the time the claimed invention was made, and does not include 
knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. 
See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). 

In response to applicant's argument that on page 11, [p][9] that the references 
fail to teach the method, wherein the step of recording frame-to-frame changes in critical 
areas of interest includes recording changes in spot area, the Examiner disagrees. 
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Black teaches a method wherein the step of recording frame-to-frame changes in critical 
areas of interest includes recording changes in spot area (see column 9, line 33-43 
where motion of spot area such as brow regions 45, see Fig 2, are recovered and 
stored in memory 20). 

In response to applicant's argument on page 1 1 [p][1 0] that the references fails to 
teach a database, that associates groups of facial movements with specific emotional 
and physical conditions of the subject; a database analysis module, for comparing the 
identified movements in the selected critical areas with the database; and a report 
generator, for generating a report of the emotional and physical condition of the subject. 
Black clearly teaches a database analysis module (26, feature and gesture recognition 
detector; see Fig 1 ) for comparing the identified movements in the selected critical 
areas with the database and Vining teaches a system for reporting findings (column 2, 
line 11-12) of an expert analysis of image data that includes a report generator (10, 
reporting system, see Fig 2) for generating a report of the emotional and physical 
condition of the subject. As to arguments for lack of motivation and improper hindsight 
reasoning, note the discussion above. 

In response to applicant's argument on page 1 1 , [p][1 1], that the references fail 
to teach the means for recording frame-to-frame changes in the critical areas includes 
means for recording changes in area, the Examiner disagrees. Black clearly teaches 
where motion of spot area such as brow regions 45, see Fig 2, are recovered and 
stored in memory 20, column 9, line 33-43. 
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In response to applicant's argument on pages 12, [p][13-14], that Lee's reference 
fail to teach the step of recording frame-to-frame changes in critical areas of interest 
includes recording changes in axial distance, to facilitate detection of axial pulsing 
movements, the Examiner disagrees. Neither Black, Pantic nor Vining teach a method 
wherein the step of recording frame-to-frame changes in critical areas of interest 
includes recording changes in axial distance to facilitate detection of axial pulsing 
movements. Therefore the Examiner added Lee's references to cure the deficiencies of 
the reference. Lee teaches a method of obtaining iris images (column 4, line 61 ) 
including the step of recording changes in axial distance (measuring the distance 
between a user's face and a camera; column 8, line 10-1 1) to facilitate detection of axial 
pulsing movements. Applicant specifically argued that Lee teaches detecting the 
movement of the entire head and not the axial pulsing distance. Examiner disagrees 
because the axial pulsing distance can be broadly interpreted as the distance between 
an individual's head a camera. 

In response to applicant's argument on pages 12-13, [p][15] and page that the 
references fail to teach, the optical imaging device includes means for measuring axial 
distance to a critical area of the face; and the means for recording frame-to-frame 
changes in critical area positions includes means for recording changes in axial 
distance, to facilitate detection of axial pulsing movements in a critical area, the 
Examiner disagrees. Note the discussion above and Lee teaches an apparatus for 
obtaining iris images (column 4, line 61) including means for measuring axial distance to 
a critical area of the face (703, distance measurer; Fig 12) and the means for recording 
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frame-to-frame changes in critical area positions includes the means for recording 
changes in axial distance (703, distance measurer; Fig 12), to facilitate detection of 
axial pulsing movements in a critical area. 

In response to applicant's argument on pages 13, [p][15-16] that the Yamamoto 
reference fails cure the deficiencies of Black, Pantic and Vinning by failing to teach 
wherein the optical imaging device comprises a charged-coupled device (CCD) camera 
producing a two-dimensional image, the Examiner disagrees. Yamamoto clearly 
teaches a face-image processing apparatus (column 3, line 25) that comprises a 
charged-coupled device (CCD) camera (see Fig. 2) producing a two-dimensional image, 
and therefore cures the deficiencies of Black, Pantic and Vining. 



Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-3, 5, 6, 8, 9 and 11 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Black et al (US Patent No.: 5,774, 591) in view of Pantic (NPL 

document titled "Expert system for automatic analysis of facial expression") further in 

view of Vining et al (US Patent No.: 6,785,410). 
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As to claim 1 , Black discloses a method for sensing selected emotions (e.g. 
happiness; column 10, line 64) in a human subject (human face; column 4, line 26). 
Black teaches generating an image of substantially all of the face of a human subject 
(40, Fig 2). Black also teaches processing the image to identify movements (determine 
image motion, column 7, line 34) in selected critical areas of the face (e.g. eye regions; 
column 7, line 35). Black also teaches comparing the identified movements in the 
selected critical areas with a database (temporal models stored on memory 28, see 
Fig1 ) that associates movements in selected critical areas with specific emotional (e.g. 
anger expression can be characterized by an initial flattening of the lip followed by a 
long downward curvature, followed by the mouth going back to a relax position; column 
20, lines 36-40) and physical conditions. 

However, Black does not teach associating movement in critical areas with 
physical conditions. Pantic teaches a method of analyzing non-verbal communication 
(page 881, [p][3], lines 4-5). Pantic also teaches associating movement in critical areas 
(specific facial actions; page 881, [p][3], line 10) with physical conditions (physiological 
reaction, page 881, [p][3], line 13). Therefore it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have added the method of 
analyzing non-verbal communication of Pantic to the method of recognizing facial 
expression of Black for vision based gesture analysis which includes face action 
recognition and emotional classification of facial expressions (page 903, [p][3], lines 2- 

5 >- 
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Note the discussion above, both Black and Pantic does not teach generating a 
report of the emotional and physical condition of the subject. Vining teaches a method 
of reporting findings (column 2, line 11-12) of an expert analysis of image data that 
includes the step of generating a report (column 3, line 37-38) of the emotional and 
physical condition of the subject. Therefore it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have added the method of 
reporting findings of Vining to the method of recognizing facial expression of Black as 
modified by Pantic for creating a report from a database of expert findings so that "data 
mining and other analysis may be conducted" (column 3, lines 39-46). 

As to claim 2, Black teaches wherein the processing step comprises 
inputting a two-dimensional frame of the image (column 7, line 47). Black teaches 
scanning the image to locate the subject's face and determine it's relative position and 
extent (column 7, line 59). Black also teaches scanning the facial part of the image to 
detect the selected critical areas (column 7, line 59). Black further teaches repeating the 
preceding steps for a sequence of image frames (column 8, lines 41-45). Additionally, 
Black teaches recording frame-to-frame changes (recover motion parameters; column 
8, line 56) in critical areas of interest. Black further teaches recording frame-to-frame 
changes in critical area positions for purposes of tracking the positions while permitting 
limited movement of the subject (column 8, lines 59-60). 

As to claim 3, Black teaches a method wherein the step of recording frame-to- 
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frame changes in critical areas of interest includes recording changes in spot area (see 
column 9, line 33-43 where motion of spot area such as brow regions 45, see Fig 2, are 
recovered and stored in memory 20). 

As to claim 5, note the discussion above, Pantic teaches wherein the comparing 
step makes use of a database that uses the facial action coding system (FACS) (page 
901, [p][1] lines 6-7). 

As to claim 6, Black teaches an apparatus for sensing selected emotions (facial 
gesture and recognition system; column 6, line 68 and column 7, line 1) in a human 
subject. Black also teaches optical imaging device (see column 7, line 44 where image 
acquisition system 8, see Fig 1 , includes a camera) for generating an image of 
substantially all of the face of a human subject. Black further teaches an image 
processing module (16, see Fig 1) for processing the image to identify movements in 
selected critical areas of the face; a database (temporal models are stored in memory 
28) associates groups of facial movements with specific emotional (e.g. anger 
expression can be characterized by an initial flattening of the lip followed by a long 
downward curvature, followed by the mouth going back to a relax position; column 20, 
lines 36-40) and physical conditions of the subject; 

However, Black does not teach associating movement in critical areas with 
physical conditions. Pantic teaches a system for analyzing non-verbal communication 
(page 881 , [p][3], lines 4-5). Pantic also teaches associating movement in critical areas 
(specific facial actions; page 881, [p][3], line 10) with physical conditions (physiological 
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reaction, page 881, [p][3], line 13). Therefore it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have added the system of 
analyzing non-verbal communication of Pantic to the facial gesture and gesture 
recognition system of Black for vision based gesture analysis which includes face action 
recognition and emotional classification of facial expressions (page 903, [p][3], lines 2- 
5). 

Black further teaches a database analysis module (26, feature and gesture 
recognition detector; see Fig 1 ) for comparing the identified movements in the selected 
critical areas with the database. 

Note the discussion both Black and Pantic does not teach a report generator, for 
generating a report of the emotional and physical condition of the subject. Vining 
teaches a system for reporting findings (column 2, line 11-12) of an expert analysis of 
image data that includes a report generator (10, reporting system, see Fig 2) for 
generating a report of the emotional and physical condition of the subject. Therefore it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to have added the system for reporting findings of Vining to the recognizing facial 
expression system of Black as modified by Pantic for creating a report from a database 
of expert findings so that "data mining and other analysis may be conducted" (column 3, 
lines 39-46). 

As to claim 8, Black teaches wherein the image processing modules comprises: 
the means for inputting a two-dimensional frame of the image is image acquisition 
system 8, see Fig 1 . 
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the means for scanning the image to locate the subject's face and determine it's relative 
position and extent is image segmentation 12, see Figl 

the means for scanning the facial part of the image to detect the critical areas of interest 
is image segmentation 12, see Figl. 

the means for repeating the preceding steps for a sequence of image frames is motion 
estimation system 16, see Figl 

the means for recording frame-to-frame changes in the critical areas of interest is 
motion estimation system 16, see Figl 

the means for recording frame-to-frame changes in critical area positions, for purposes 
of tracking the positions while permitting limited movement of the subject is region 
tracking system, 1 8, see Fig 1 

As to claim 9, Black teaches an apparatus wherein the means for recording 
frame-to-frame changes in the critical areas includes means for recording changes in 
area (see column 9, line 33-43 where motion of spot area such as brow regions 45, see 
Fig 2, are recovered and stored in memory 20). 

As to claim 1 1 , note the discussion above, Pantic teaches wherein the database uses 
the facial action coding system (FACS) (page 901 , [p][1 ] lines 6-7). 

3. Claims 4 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Black et al (US Patent No.: 5,774, 591) in view of Pantic (NPL document titled "Expert 
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system for automatic analysis of facial expression") further in view of Vining et al (US 
Patent No.: 6,785,410) further in view of Lee et al (US Patent No. 7,095,901 ). 

As to claim 4, neither Black, Pantic nor Vining teach a method wherein the step 
of recording frame-to-frame changes in critical areas of interest includes recording 
changes in axial distance to facilitate detection of axial pulsing movements. Lee teaches 
a method of obtaining iris images (column 4, line 61 ) including the step of recording 
changes in axial distance (measuring the distance between a user's face and a camera; 
column 8, line 10-11) to facilitate detection of axial pulsing movements. Therefore it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to have added the method of obtaining iris images of Lee to the method of 
recognizing facial expression of Black as modified by Pantic and Vining for measuring 
the distance between a user and a "camera more accurately, more conveniently and 
more quickly and to focus the camera automatically" (column 4, lines 2-4). 

As to claim 10, neither Black, Pantic nor Vining teach an apparatus wherein the 
optical imaging device includes means for measuring axial distance to a critical area of 
the face; and the means for recording frame-to-frame changes in critical area positions 
includes means for recording changes in axial distance, to facilitate detection of axial 
pulsing movements in a critical area. Lee teaches an apparatus for obtaining iris 
images (column 4, line 61) including means for measuring axial distance to a critical 
area of the face (703, distance measurer; Fig 12) and the means for recording frame-to- 
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frame changes in critical area positions includes the means for recording changes in 
axial distance (703, distance measurer; Fig 12) ', to facilitate detection of axial pulsing 
movements in a critical area. Therefore it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to have added the apparatus for 
obtaining iris images of Lee to the facial gesture and gesture recognition system of 
Black as modified by Pantic and Vining for measuring the distance between a user and 
a "camera more accurately, more conveniently and more quickly and to focus the 
camera automatically" (column 4, lines 2-4). 

4. Claim 7 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Black et 
al (US Patent No.: 5,774, 591) in view of Pantic (NPL document titled "Expert system for 
automatic analysis of facial expression") further in view of Vining et al (US Patent No.: 
6,785,410) further in view of Yamamoto (US Patent No.: 6,549,664). 

As to claim 7, note the discussion, neither Black, Pantic nor Vining teach an 
apparatus wherein the optical imaging device comprises a charged-coupled device 
(CCD) camera producing a two-dimensional image. Yamamoto teaches a face-image 
processing apparatus (column 3, line 25) that comprises a charged-coupled device 
(CCD) camera (see Fig. 2) producing a two-dimensional image. Therefore it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
have substituted the optical image device of Black as modified by Pantic and Vining with 
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the face-image processing apparatus of Yamamoto for photographing of a person's face 
used for extracting a certain characteristic such as an eye or nostril (column 3, lines 29- 

35). 

Rejections for New Claims 

Claims 12-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Black et al (US Patent No.: 5,774, 591) in view of Pantic (NPL document titled "Expert 
system for automatic analysis of facial expression") further in view of Vining et al (US 
Patent No.: 6,785,410) further in view of Stirbl et al (US Patent No.: 5,507,291). 

As to claim 12, note the discussion above, neither Black, Pantic or Vining 
discloses the method wherein generating the image comprises implementing electronic 
speckle pattern interferometry to generate a speckle pattern of diffusely reflected 
coincident light that corresponds to the face of the human subject. Stirbl teaches a 
method for for remotely determining an individual emotional (column 1, lines 8-10) state 
that includes generating the image comprises implementing electronic speckle pattern 
interferometry to generate a speckle pattern of diffusely reflected coincident light that 
corresponds to the face of the human subject (note that the method can determine the 
physical or physiological parameter from the iris (see abstract), thus a speckle pattern of 
the face can be obtained). At the time of the invention, it would have been obvious to a 
person of ordinary skill in the art to have combined the teachings of Stirbl with that of 
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Black as modified by Pantic and Vining for obtaining information pertinent to a person's 
emotional state, without the person's knowledge, thereby determining the truthfulness 
or sincerity of a person during an interview (column 1 , lines 42-58). 

As to claim 1 3, note the discussion above, Stirbl teaches the method wherein 
processing the images comprises identifying fluctuations in multiple-pixel reflectivity of 
the speckle-spot pattern compared with non-vibratory areas of adjacent facial surfaces 
to identify the selected critical areas of the face (see column 6, lines 45-50). 

As to claim 14, note the discussion above, Stirbl teaches the method wherein 
processing the image comprises tracking the recording frame-to-frame changes in at 
least one of position, size and intensity of speckle-spots in the selected critical areas of 
the speckle-spot pattern (note that changes in selected target point of an individual can 
be tracked, see column 56-56). 

As to claim 1 5, note the discussion above, Stirbl teaches the method wherein 
generating comprises: obtaining a first image of substantially all of the face of the 
human subject at a beginning of a pulse period associated with a pulsed light source; 
obtaining a second image of substantially all of the face of the human subject at a 
beginning of a pulse period associated with a pulsed light source (166, see Fig 5, note 
that the light source is modulated); and subtracting the second image from the first 
image to generate a resulting image of substantially all of the face of the human 
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subjects having a high contrast ratio (see column change in intensity is determined, see 
column 2, lines 56-67). 

Claim 16 differ from claim 12 only in that claim 12 is a method claim whereas, 
claim 16 is an apparatus claim. Thus, claim 16 is analyzed as previously discussed with 
respect to claim 12 above. 



Claim 17 differ from claim 13 only in that claim 13 is a method claim whereas, 
claim 17 is an apparatus claim. Thus, claim 17 is analyzed as previously discussed with 
respect to claim 13 above. 

Claim 18 differ from claim 14 only in that claim 14 is a method claim whereas, 
claim 18 is an apparatus claim. Thus, claim 18 is analyzed as previously discussed with 
respect to claim 1 4 above. 

Claim 19 differ from claim 15 only in that claim 15 is a method claim whereas, 
claim 19 is an apparatus claim. Thus, claim 19 is analyzed as previously discussed with 
respect to claim 1 5 above. 
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Conclusion 

5. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 



Inquires 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrae S. Allison whose telephone number is (571) 
270-1052. The examiner can normally be reached on Monday-Friday, 8:00 am - 5:00 
pm, EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Mancuso can be reached on (571) 272-7695. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Andrae Allison 

March 2, 2007 // 





